Light-field capture devices (also referred to as “light-field image data acquisition devices”) are defined herein as any devices that are capable of capturing light-field data, optionally processing light-field data, optionally accepting and acting upon user input, and optionally displaying or otherwise outputting images and/or other types of data.
Light-field capture devices may capture light-field data using any suitable method for doing so. One example of such a method includes, without limitation, using a microlens array on top of an image sensor (e.g., a CCD or CMOS sensor) as described in Ng et al., Light-field photography with a hand-held plenoptic capture device, Technical Report CSTR 2005-02, Stanford Computer Science. Other examples include the use of a plurality of independently controlled cameras, each with its own lens and sensor, an array of cameras that image onto a single shared sensor, a plenoptic lens, and/or any combination of these.
Light-field data may be represented or encoded in any of a number of different ways, including (but not limited to) as a 4D image, as a 2D array of 2D disk images such as described in Ng et al., as a 2D array of 2D images of a scene taken from different perspectives such as would be captured by an array of cameras (and which are known as “sub-aperture” images in Ng et al.), and as any combination of these.
Whichever representation is used, light-field data captured by a light-field capture device may be processed to produce a 2D image that is suitable for display or output. Such light-field processing can include (but is not limited to) generating refocused images, generating perspective views of a scene, generating depth maps of a scene, generating all-in-focus or extended depth of field (EDOF) images, generating parallax-shifted or perspective views of a scene, generating stereo image pairs, and/or any combination of these. Additionally, such generated 2D images may be modified or annotated based on the results of analysis of the light-field data performed by algorithms that process the captured light-field data.
Data captured by light-field capture devices contains information from which scene depths may be inferred or measured, and the range of depths captured in a scene is related to the set of possible 2D images which may be rendered from (or projected from) the captured light-field data. The “amount of refocusing”, the “3D-ness”, and the range of perspective/parallax shifting that is possible from a captured set of light-field data is, in general, proportional to the dioptric range of scene depths that were captured. However, a standard 2D preview image of a scene, as is used by conventional cameras, does not generally communicate to the user the extent to which the range of depths captured is suitable for generating compelling output images with large amounts of refocusing, 3D, parallax/perspective shifting, or any other effects that may be generated from captured light-field data.
Additionally, some features or capabilities that are commonplace in conventional cameras may not be generally available in light-field capture devices unless the captured light-field data is suitably processed. One example of such a feature is the ability to record 2D video streams on the device. Another example is the ability of the device to host applications that are able to access the camera system and which expect 2D image data to be produced by it (for example, photo and camera apps on mobile devices such as native iOS and Android camera apps, as well as third-party mobile apps such as Instagram). Conventionally, such applications may not run properly on a light-field capture device, particularly if the capture device does not properly process the captured light-field data to make it available as a conventional 2D image stream for such applications.